Bayesian network representation

Independence

A and B are independent, denoted $P ⊨ (A ⊥ B)$ , iff $P (A | B) = P (A)$ or $P (B | A) = P (B)$ or $P (A, B) = P (A) P (B)$

Conditional independence

$X$ is conditionally independent of $Y$ given $Z$ if

P (X | Y, Z) = P (X | Z) o r P (Y | X, Z) = P (Y, Z)

o r P (X, Y | Z) = P (X | Z) P (Y | Z)

Notion: $P ⊨ (X ⊥ Y ∣ Z)$

Bayes' Rule

Product rule $P (a \land b) = P (a | b) P (b) = P (b | a) P (a)$

\Rightarrow B a y e s^{'} r u l e P (a | b) = \frac{P (b | a) P (a)}{P (b)}

or in distribution form

P (Y | X) = \frac{P (X | Y) P (Y)}{P (X)} = α P (X | Y) P (Y)

Useful for assessing diagnostic probability from causal probability:

P (C a u s e | E f f e c t) = \frac{P (E f f e c t | C a u s e) P (C a u s e)}{P (E f f e c t)}

An example of naive Bayes model:
Bayesian network representation-1777308741194.webp

P (C a u s e, E f f e c t_{1}, . . ., E f f e c t_{n}) = P (C a u s e) \prod_{i} P (E f f e c t_{i} | C a u s e)

Total number of parameters is linear in $n$

Bayesian network

A simple, graphical notation for conditional independence assertions and hence for compact specification of full joint distributions.

Syntax:

a set of nodes, one per variable
a directed, acyclic graph (link ≈ "directly influences")
a conditional distribution for each node given its parents: $P (X_{i} | P a r e n t s (X_{i}))$ (CPD)

In the simplest case, conditional distribution represented as a conditional probability table (CPT) giving the distribution over $X_{i}$ for each combination of parent values

Example: Bayesian network representation-1777308964529.webp

Global semantics

Global semantics defines the full joint distribution as the product of the local conditional distributions:

P (x_{1}, \dots, x_{n}) = \prod_{i = 1}^{n} P (x_{i} | p a r e n t s (X_{i}))

active trail

Let Z be a subset of observed variables.
The trail $X_{i - 1} ⇌ X_{i} ⇌ X_{i + 1}$ is active given Z if

$\forall X_{i - 1} \to X_{i} \leftarrow X_{i + 1}$ , $X_{i}$ or one of its descendants are in Z
no other node along the trail is in Z

direct separation(d-seperation)

Two sets of nodes X, Y are d-separated given Z if there is no active trail between any $X \in X$ and $Y \in Y$ given Z

To determine if X and Y are independent given Z:

traverse the graph bottom-up marking all nodes in Z or having descendants in given Z
$2$ traverse the graph from $X$ to $Y$ , stopping if we get to a blocked node
if we can't reach Y, then X and Y are independent

A node is blocked if either the middle of an unmarked v-structure, or in Z (not both)

Local semantics

Local semantics: each node is conditionally independent of its nondescendants given its parents

Theorem: Local semantics ⇔ global semantics

Markov blanket

Each node is conditionally independent of all others given its Markov blanket: parents + children + children's parents